Exploiting Transformer in Sparse Reward Reinforcement Learning for Interpretable Temporal Logic Motion Planning

Automaton based approaches have enabled robots to perform various complex tasks. However, most existing automaton algorithms highly rely on the manually customized representation of states for considered task, limiting its applicability in deep reinforcement learning algorithms. To address this issue, by incorporating Transformer into learning, we develop a Double-Transformer-guided Temporal Logic framework (T2TL) that exploits structural feature twice, i.e., first encoding LTL instruction via module efficient understanding task instructions during training and then context variable again improved performance. Particularly, is specified co-safe LTL. As semantics-preserving rewriting operation, progression exploited decompose learnable sub-goals, which not only converts non-Markovian reward decision processes Markovian ones, but also improves sampling efficiency simultaneous multiple sub-tasks. An environment-agnostic pre-training scheme further incorporated facilitate resulting an The simulation results demonstrate effectiveness T2TL framework.